Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications

摘要

Facial expressions are an important way through which humans interactsocially. Building a system capable of automatically recognizing facialexpressions from images and video has been an intense field of study in recentyears. Interpreting such expressions remains challenging and much research isneeded about the way they relate to human affect. This paper presents a generaloverview of automatic RGB, 3D, thermal and multimodal facial expressionanalysis. We define a new taxonomy for the field, encompassing all steps fromface detection to facial expression recognition, and describe and classify thestate of the art methods accordingly. We also present the important datasetsand the bench-marking of most influential methods. We conclude with a generaldiscussion about trends, important questions and future lines of research.